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feature 


Trust-Based  Analysis  of  an 
Air  Force  Collision  Avoidance 
System 


Engaging  test  pilots  in  the  analysis 
reveals  the  qualities  of  the  system 
that  can  lead  to  increased  trust  and, 
as  a  result,  fewer  crashes  into  the 
terrain. 


FEATURE  AT  A  GLANCE: 

This  case  study  analyzes  the 
factors  that  influence  trust 
and  acceptance  among  users 
(in  this  case,  test  pilots)  of  the 
Air  Force’s  Automatic  Ground 
Collision  Avoidance  System. 

Our  analyses  revealed  that 
test  pilots’  trust  depended  on 
a  number  of  factors,  including 
the  development  of  a  nuisance- 
free  algorithm,  designing  fly-up 
evasive  maneuvers  consistent 
with  a  pilot’s  preferred  behavior, 
and  using  training  to  assess, 
demonstrate,  and  verify  the 
system’s  reliability.  These  factors 
are  consistent  with  the  literature 
on  trust  in  automation  and 
could  lead  to  best  practices  for 
automation  design,  testing,  and 
acceptance. 

KEYWORDS: 

trust  in  automation,  automation 
reliance,  CFIT,  nuisance 
budget,  test  pilot,  Auto-GCAS 
technology,  design,  automation 
design,  human-machine 
interaction,  automated  system 


By  Joseph  B.  Lyons,  Nhut  T.  Ho,  Kolina  S.  Koltai,  Gina  Masequesmay,  Mark  Skoog, 
Artemio  Cacanindin,  &  Walter  W.  Johnson 


In  military  aviation,  controlled  flight  into 

terrain  (CFIT)  is  a  significant  cause  of  the 
loss  of  life  for  pilots  (Richardson,  Eger, 

&  Hamilton,  2015).  CFIT  occurs  when  a 
properly  functioning  aircraft  collides  with 
terrain  because  the  pilot  is  unaware  of  or 
unable  to  avoid  the  danger  before  it  is  too 
late.  These  collisions  may  occur  because 
of  the  pilot’s  spatial  disorientation  (which 
is  a  cognitive  precursor  to  CFIT)  or  from 
G-force-induced  loss  of  consciousness. 

To  mitigate  the  precursors  of  CFIT,  the 
Air  Force  and  its  collaborators  in  the  past 
three  decades  have  developed  the  Automatic 
Ground  Collision  Avoidance  System  (Auto- 
GCAS),  which  has  been  successfully  flight- 
tested  and  is  being  integrated  into  and  used 
in  operational  F-16  aircraft.  Auto-GCAS 
is  a  technology  that  assumes  control  of  an 
aircraft  when  an  imminent  collision  with  the 
ground  is  detected  and  returns  control  back 
to  the  pilot  when  the  collision  is  averted. 

We  describe  a  case  study  that  analyzed 
Auto-GCAS  from  a  human-machine  trust 
perspective.  In  particular,  we  explain  the 
role  of  trust  as  related  to  the  development 
and  acceptance  of  this  technology  by 
discussing  the  critical  drivers  of  trust  from 
the  perspective  of  test  pilots  who  have 
extensive  experience  with  Auto-GCAS. 

We  conclude  with  recommendations  for 
automated  system  design  to  foster  trust  of 
future  automated  systems. 

WHY  TRUST  MATTERS 

Trust,  one’s  willingness  to  accept 
vulnerabilities  in  relation  to  another 
entity  (i.e.,  technology),  is  a  critical  driver 
of  human-machine  interactions  (Chen 
&  Barnes,  2014;  Lee  &  See,  2004;  Lyons 
&  Stokes,  2012).  An  individual’s  trust  of 


technology  will  influence  how  much  he  or 
she  will  accept  that  technology  and  rely  on  it 
(Lee  &  See,  2004). 

Pilots’  acceptance  of  Auto-GCAS  has 
been  an  ongoing  concern  (Richardson 
et  al.,  2015)  because  it  is  a  system  that  takes 
control  away  from  the  pilot,  albeit  briefly. 
Understanding  pilot  trust  of  Auto-GCAS 
is  critical  to  its  operational  performance 
because  pilots  have  the  option  to  turn  the 
system  on  or  off  during  operations. 

Although  Auto-GCAS  was  designed  to 
prevent  up  to  98%  of  historical  incidents, 
and  to  work  only  in  situations  when  the 
pilot  is  experiencing  spatial  disorientation  or 
G-force-induced  loss  of  consciousness,  pilots 
can  misuse  the  system  if  they  overtrust  it 
and  believe  that  Auto-GCAS  will  always  save 
them.  This  misbelief  can  motivate  the  pilot 
to  fly  more  aggressively  or  brazenly  and  to 
misuse  Auto-GCAS  as  a  combat  tool.  Thus 
understanding  pilot  trust  is  a  crucial  step  in 
preventing  pilot  complacency  (i.e.,  misuse). 

The  vast  majority  of  studies  focusing  on 
human-machine  trust  have  been  conducted 
in  laboratory  settings.  These  studies,  along 
with  limited  field  studies,  have  identified 
a  number  of  consistent  predictors  of  trust 
in  automation  (for  reviews  on  the  topic, 
see  Hoff  &  Bashir,  2015;  Lee  &  See,  2004; 
Onnasch,  Wickens,  Li,  &  Manzey,  2014; 

Rice,  2009).  Key  findings  from  this  literature 
show  that  factors  such  as  high  reliability, 
low  error  rates  (particularly  low  false- 
alarm  rates),  transparency,  familiarity,  and 
anthropomorphic  features  increase  trust 
(Hoff  &  Bashir,  2015). 

Similarly,  Lee  and  See  (2004)  suggested 
that  to  increase  trust,  designers  should  show 
the  system’s  past  performance  and  how  it 
works,  simplify  algorithms  to  make  them 
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understandable,  highlight  the  system’s  intent,  and  conduct 
training  to  demonstrate  the  system’s  reliability.  Research  also 
shows  that  in  addition  to  performance-based  beliefs,  social 
beliefs  and  institutional  norms  are  predictive  of  early  trust 
of  new  technologies  (Li,  Hess,  &  Valacich,  2008);  thus  it  is 
important  to  capture  the  full  spectrum  of  trust  antecedents 
during  technology  evaluations  with  users.  Yet,  it  is  unknown 
how  well  these  factors  will  map  onto  trust  of  test  pilots  using 
an  actual  automated  system  that  employs  a  high  level  of 
automation  during  high-risk  operations. 

DESIGN  MOTIVATIONS  FOR  AUTO-GCAS  THAT  IMPACT 
PILOT  TRUST 

Auto -GCAS  was  designed  with  three  ranked  design  principles 
meant  to  guide  the  development  and  acceptance  of  the  system: 

(1)  do  no  harm,  which  requires  that  Auto-GCAS  not  cause 
any  harm  to  the  pilot  or  the  aircraft;  (2)  do  not  impede,  which 
requires  the  system  to  be  nuisance  free  and  thus  not  interfere 
with  the  mission;  and  (3)  avoid  collision,  which  requires  the 
system  to  avoid  collision  with  terrain  (Richardson  et  al.,  2015). 
The  principal  goal  of  Auto-GCAS  is  to  mitigate  the  problem  of 
CFIT.  It  is  designed  to  do  so  by  taking  over  control  of  the  aircraft, 
maneuvering  it  away  from  danger,  and  returning  control  back 
to  the  pilot.  This  functionality  can  be  contrasted  with  previous 
collision  avoidance  technologies,  which  primarily  used  warning 
systems  and  proved  insufficient  for  mitigating  CFIT  (Richardson 
et  al.,  2015). 

Warning-based  systems  were  not  an  ideal  solution  to  the 
CFIT  problem  because  (a)  they  are  prone  to  false  alarms,  which 
can  degrade  trust  (Geels-Blair,  Rice,  &  Schwark,  2013);  (b)  they 
require  the  user  to  manually  respond  and  thus  are  not  effective 
when  the  pilot  is  incapacitated  or  spatially  disoriented;  and 
(c)  the  pilot  may  not  always  correctly  recognize  a  warning  or 
correctly  make  the  terrain  collision  evasion  maneuver. 

To  overcome  these  limitations,  Auto-GCAS  was  developed 
with  a  number  of  innovative  approaches  and  solutions.  To 
address  the  nuisance  problem,  a  nuisance  threshold  was 
determined  that  ensured  that  in  the  pilot’s  view,  Auto-GCAS 
would  not  trigger  too  early.  This  solution  was  accomplished 
by  conducting  a  flight-test  program  to  define  what  would 
be  the  maximum  acceptable  time  before  CFIT  to  initiate  a 
recovery  maneuver  that  the  pilot  would  not  consider  to  be 
a  nuisance.  Considering  only  this  problem  from  a  temporal 
perspective  was  unique,  as  previous  efforts  had  used  distance 
above  the  ground  as  a  metric  for  nuisance. 

To  determine  the  time  to  CFIT  at  which  a  fly- up  maneuver 
would  not  be  a  nuisance,  pilots  flew  aircraft  toward  terrain  in 
different  conditions  and  manually  initiated  a  recover  as  they 
reached  their  own  comfort  threshold.  They  then  rated  their 
anxiety  level  at  recovery  initiation  to  capture  whether  they  had 
accidentally  initiated  the  recovery  sooner  or  later  than  they  had 
intended.  The  duration  between  the  point  at  which  a  recovery 
initiation  would  no  longer  be  considered  a  nuisance  and  the 
point  at  which  the  aircraft’s  maximum  performance  recovery 
would  just  clear  the  ground  is  called  a  nuisance  budget. 


Figure  1 .  Head-up  display  image  with  chevrons. 

Auto-GCAS  was  also  designed  with  a  head- up  display 
(HUD)  anticipation  cue  for  potential  activation.  Providing 
human  operators  with  transparency  into  the  system’s  capabilities, 
purpose,  intent,  and  analytical  underpinnings  is  another  way 
to  foster  trust  (Lyons,  2013).  In  the  Auto-GCAS  system, 
operators  are  able  to  see  two  chevrons  (i.e.,  the  two  arrows 
shown  in  Figure  1)  on  the  HUD  that  dynamically  intersect 
as  the  aircraft  approaches  a  possible  CFIT  situation.  These 
chevrons  allow  the  pilot  to  anticipate  when  the  system  is 
about  to  engage. 

TEST  PILOT  EVALUATION 

Our-research  occurred  in  the  context  of  a  larger  case  study 
of  the  Auto-GCAS  technology  involving  engineers,  test 
pilots,  and  managers  engaged  in  the  Auto-GCAS  program. 
This  article  focuses  solely  on  the  test-pilot  portion  of  that 
case  study,  and  specifically  the  factors  that  influenced  test- 
pilot  trust.  Our  general  strategy  for  assessing  test-pilot  trust 
of  Auto-GCAS  involved  a  qualitative  review  of  the  factors 
that  influenced  their  trust  development  of  the  system.  This 
assessment  was  accomplished  through  survey  and  interview 
techniques. 

First,  test  pilots  responded  to  an  online  survey  that  assessed 
their  general  attitudes  toward  Auto-GCAS,  initial  perceptions 
of  the  system,  experiences  with  the  system,  and  open-ended 
questions  about  the  factors  that  influenced  their  trust  or 
distrust  with  the  system.  (The  majority  of  questions  in  the 
online  survey  involved  perceptions  of  engineers  and  managers 
associated  with  the  Auto-GCAS  program  and  are  out  of 
the  scope  of  this  article.)  The  survey  data  formed  the  basis 
for  follow-up  interview  questions  relating  to  what  factors 
influenced  the  pilots’  trust  over  time. 

Experimental  test  pilots  (N  -  15)  from  Edwards  Air 
Force  Base  participated  in  a  serious  of  interviews.  Two  to 
four  interviewers  were  present,  and  the  interviews  lasted 
approximately  20  to  60  min,  depending  on  the  availability  of 
the  pilots.  The  interview  focused  on  revealing  pilot  attitudes 
toward  Auto-GCAS,  identifying  the  trust  levels  among  test 
pilots  of  Auto-GCAS,  and  discussing  reasons  for  trust  or 
distrust  of  Auto-GCAS.  The  interviews  were  recorded  and 
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transcribed  using  NVivo  software.  The  interview  team  then 
reviewed  the  files  and  looked  for  common  themes  and  trends 
using  a  grounded-theory  approach. 

SUBSET  OF  FACTORS  SHAPING  TEST  PILOT  TRUST 
OF  AUTO-GCAS 

The  following  sections  represent  the  factors  that  influenced 
trust  of  Auto-GCAS  among  the  test  pilot  community. 

Avoiding  false  alarms .  The  primary  driver  of  trust  among  test 
pilots  was  the  demonstrated  ability  of  Auto-GCAS  to  avoid 
false  alarms  (i.e.,  the  creation  of  the  nuisance  budget).  The 
nuisance  criteria  were  found  to  be  governed  by  two  principles 
for  an  avoidance  maneuver  to  be  acceptable  to  a  pilot:  It  must 
be  both  aggressive  and  timely  -  aggressive  in  the  sense  that 
the  avoidance  is  using  a  significant  amount  of  the  aircraft’s 
available  maneuvering  capability,  and  timely  in  the  sense  that 
it  does  not  begin  the  maneuver  so  early  that  it  is  deemed  a 
nuisance  or  so  late  that  the  aircraft  does  not  have  sufficient 
maneuverability  to  avoid  hitting  the  surrounding  terrain. 

The  pilots  apply  a  safety  buffer  based  on  their  uncertainty 
of  the  situation  (rapidly  changing  aircraft  attitude  or  conditions, 
high  turbulence,  poor  visibility,  etc.)  to  determine  when  the 
aggressive  avoidance  maneuver  should  be  initiated.  Because 
the  aggressive  portion  of  the  criteria  embeds  within  it  the 
aerodynamics  unique  to  a  vehicle,  the  timely  portion  of 
the  criteria  remains  the  same  for  all  aircraft.  We  also  found 
that  the  timely  criterion  could  be  approximated  by  a  single 
equation,  with  the  maximum  acceptable  time  being  the  point 
at  which  the  aggressive  recovery  will  just  miss  the  ground. 

This  finding  is  consistent  with  research  on  trust  in  automation 
showing  how  false  alarms  can  degrade  trust  (Geels-Blair  et  al., 
2013;  Hoff  &  Bashir,  2015;  Rice,  2009). 

Performance  of  the  system .  A  second  factor  driving  test-pilot 
trust  was  the  knowledge  of  the  system’s  reliability.  Auto- 
GCAS  was  designed  to  approach  98%  reliability  in  preventing 
CFIT.  The  test  pilots  experienced  this  high  reliability  in 
operational  testing,  which  was  clearly  to  the  benefit  of  their 
trust  in  Auto-GCAS.  This  finding  is  consistent  with  prior 
research  showing  that  performance  is  a  significant  predictor 
of  human-machine  trust  (Hancock  et  al.,  2011;  Hoff  &  Bashir, 
2015;  Lee  &  See,  2004). 

Transparency .  Automation  transparency,  in  the  form  of 
chevrons,  was  an  important  topic  that  emerged  in  this  study 
because  it  affects  how  pilots  develop  and  calibrate  their  trust 
of  Auto-GCAS.  The  test  pilots  reported  that  the  chevrons 
enabled  them  to  better  anticipate  the  behavior  of  Auto-GCAS, 
consistent  with  prior  views  of  transparency  (Lyons,  2013). 

However,  we  must  approach  this  issue  with  some  degree 
of  caution  given  the  context  of  Auto-GCAS.  It  is  possible, 
for  instance,  given  that  Auto-GCAS  is  a  safety  system,  that 
pilots  (who  are  faced  with  situations  that  require  intense  aerial 
maneuvers)  could  use  the  chevrons  to  operate  their  aircraft  with 


greater  risk,  as  the  chevrons  provide  enhanced  awareness  of  one’s 
relative  proximity  to  the  ground.  Such  behaviors  would  conflict 
with  the  overall  intent  and  design  of  Auto-GCAS,  both  of  which 
assume  that  pilots  will  fly  their  aircraft  in  the  same  manner  that 
they  had  prior  to  the  availability  of  Auto-GCAS.  Authors  of 
future  research  should  explore  the  role  of  transparency  within 
safety  systems,  specifically  within  the  context  of  Auto-GCAS. 

Familiarity  of  the  maneuver.  Test  pilots  reported  strong 
positive  perceptions  of  the  Auto-GCAS  maneuvering 
capabilities.  Auto-GCAS  was  designed  to  engage  a  wings- 
level,  5-G  pull-up,  which  is  consistent  with  pilot  training 
and  behavior.  This  familiarity  supported  stronger  trust 
perceptions  among  the  test  pilots. 

CONCLUSION  AND  LESSONS  LEARNED 

In  this  article,  we  discussed  the  Air  Force’s  Auto-GCAS 
system  and  highlighted  several  of  the  drivers  of  trust  of  this 
system  from  the  perspective  of  test  pilots.  All  these  factors 
correspond  to  variables  noted  in  the  trust  literature;  from  a  real- 
life  context  in  which  humans  interact  with  automation,  they 
lend  applied  support  to  the  literature.  Additionally,  we  described 
a  few  design  features  used  by  the  Auto-GCAS  program  that 
resulted  in  higher  trust  perceptions  among  test  pilots.  Although 
the  current  results  are  qualitative  in  nature,  they  provide  a  rich 
set  of  inputs  from  an  operational  community  regarding  the 
factors  that  shape  trust  in  automation. 

The  lessons  learned  in  conducting  this  research  are  both 
methodological  and  design  based  in  nature.  The  study 
reinforced  the  notion  that  flexibility  and  access  to  applied 
participants  are  keys  to  success  in  researching  applied 
systems.  The  test  pilots  had  rigorous  schedules,  and  the 
research  team  had  to  adapt  to  the  pilots’  constraints.  Second, 
none  of  this  research  is  possible  without  the  support  of  key 
stakeholders  to  foster  access  to  the  applied  participants. 

From  a  design  standpoint,  it  is  clear  that  the  issue  of 
false  alarms  was  paramount  to  the  pilots.  These  individuals 
are  at  the  “tip  of  the  spear”  of  military  operations,  and  the 
last  thing  they  wanted  was  interference  from  an  automated 
system.  Once  these  concerns  were  subdued,  the  test  pilots 
became  much  more  accepting  of  the  technology.  This  finding 
reinforces  the  importance  of  user-centered  design,  as  the 
guiding  principle  for  the  collision  avoidance  algorithm  was 
based  on  the  pilot’s  nuisance  budget. 

A  second  lesson  learned  involves  giving  users  an 
opportunity  to  experience  the  system  in  action,  as  this 
appeared  to  be  a  key  driver  of  test-pilot  trust  of  the  system. 
Third,  designing  the  system  to  provide  cues  related  to 
activations  was  useful.  This  form  of  real-time  transparency 
was  an  important  driver  of  trust  in  our  study.  Finally,  the 
maneuver  designed  into  the  system  was  one  familiar  to  pilots, 
which  also  contributed  to  their  trust  of  the  system.  Designers 
of  future  automated  systems  might  consider  incorporating 
these  elements  into  their  design  process,  lest  they  be  faced 
with  a  distrustful  user  base. 
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